With increasing instrumentation and increasing automation in public sectors come ever increasing amounts of data and the desire to use that data for uses such as improving automation and, more generally, increasing realtime awareness of information that can be useful for decision-makers. For example, cities are becoming increasingly instrumented with a variety of sensors, such as magnetic loop detectors, infra-red sensors, pressure pads, roadside radar, and web cameras. Many of these sensors can provide relevant information regarding traffic flows in the city, like vehicle speed, count, and types. In particular, the increasing availability of video cameras installed at intersections of urban streets or other roads in the city, allows for the extraction of realtime estimations of traffic flows per type of vehicle, such as the flow rate of yellow cabs. The citywide web cameras capture traffic video twenty four/seven, continuously, generating large-scale traffic video data. These cameras either are low quality, or it may be the intent to only process low quality versions of their video. This precludes most existing techniques for traffic flow analysis.
It is expected that driverless vehicles will increasingly populate city streets and become a significant fraction of traffic flows in the near future. While driverless-vehicle sensing provides awareness about the local conditions where it operates, infrastructure sensors, increasingly available with the emergence of the Internet of things, have the potential to provide global awareness for driverless cars of traffic conditions across an entire city or other area. Much effort has been on processing sensing data collected by the driverless car's suite of sensors. However, much less work exists on processing simultaneously streaming video from city traffic cameras to build a multi-model autonomous system. Indeed, problems inherent in typical traffic cameras, such as low framerate and low resolution, along with problems inherent in the nature of vehicle traffic, such as varying weather conditions, daily changing lighting conditions, and wide variety of vehicle types and models, make using information from such cameras in any meaningful way extremely challenging.